
mdrc 

BUILDING KNOWLEDGE 
TO IMPROVE SOCIAL POLICY 


JUNE 2012 


What Can a 
Multifaceted 
Program 
Do for 
Community 
College 
Students? 


EARLY RESULTS FROM 
AN EVALUATION OF 
ACCELERATED STUDY IN 
ASSOCIATE PROGRAMS (ASAP) 
FOR DEVELOPMENTAL 
EDUCATION STUDENTS 


Susan Scrivener 
Michael j. Weiss 
Colleen Sommo 



What Can a Multifaceted Program Do for 
Community College Students? 


Early Results from an Evaluation of 
Accelerated Study in Associate Programs (ASAP) 
for Developmental Education Students 


Susan Scrivener 
Michael J. Weiss 
Colleen Sommo 

with 

Hannah Fresques 


June 2012 


mdrc 

BUILDING KNOWLEDGE 
TO IMPROVE SOCIAL POLICY 


The ASAP evaluation is supported by the Leona M. and Harry B. Helmsley Charitable Trust and 
the Robin Hood Foundation. 


Dissemination of MDRC publications is supported by the following funders that help finance 
MDRC’s public policy outreach and expanding efforts to communicate the results and implica- 
tions of our work to policymakers, practitioners, and others: The Annie E. Casey Foundation, The 
George Gund Foundation, Sandler Foundation, and The Starr Foundation. 

In addition, earnings from the MDRC Endowment help sustain our dissemination efforts. Con- 
tributors to the MDRC Endowment include Alcoa Foundation, The Ambrose Monell Foundation, 
Anheuser-Busch Foundation, Bristol-Myers Squibb Foundation, Charles Stewart Mott Founda- 
tion, Ford Foundation, The George Gund Foundation, The Grable Foundation, The Lizabeth and 
Frank Newman Charitable Foundation, The New York Times Company Foundation, Jan Nichol- 
son, Paul H. O’Neill Charitable Foundation, John S. Reed, Sandler Foundation, and The Stupski 
Family Fund, as well as other individual contributors. 


The findings and conclusions in this report do not necessarily represent the official positions or 
policies of the funders. 


For information about MDRC and copies of our publications, see our Web site: www.mdrc.org. 
Copyright © 2012 by MDRC.® All rights reserved. 



Overview 


In recent years, there has been unprecedented national focus on the importance of increasing the 
stubbornly low graduation rates of community college students. Most reforms that have been tried 
are short-term and address one or only a few barriers to student success. The City University of New 
York’s (CUNY’s) Accelerated Study in Associate Programs (ASAP), launched in 2007 with 
funding from Mayor Bloomberg’s Center for Economic Opportunity (CEO), is an uncommonly 
multifaceted and long-term program designed to help community college students graduate. 

ASAP requires students to attend college full time and provides a rich array of supports and incen- 
tives for up to three years, with a goal of graduating at least 50 percent of students within three years. 
Unlike many programs, ASAP aims to simultaneously address multiple barriers to student success 
over many semesters. The program model includes some block-scheduled classes for ASAP 
students for the first year of the program; an ASAP seminar for at least the first year, which covers 
such topics as goal-setting and academic planning; comprehensive advisement; tutoring; career 
services; a tuition waiver that covers any gap between a student’s financial aid and tuition and fees; 
free MetroCards for use on public transportation; and free use of textbooks. 

This report presents very promising early findings from a random assignment study of ASAP at 
three CUNY community colleges: Borough of Manhattan, Kingsborough, and LaGuardia. For the 
study, ASAP targets low-income students who need one or two developmental (remedial) courses to 
build their reading, writing, or math skills. The study compares ASAP with regular services and 
classes at the colleges. Key findings include effects on: 

• Full-time enrollment. During the study’s first semester, ASAP increased full-time enrollment 
by 1 1 percentage points: 96 percent of the students assigned to ASAP enrolled full time, com- 
pared with 85 percent of the comparison group. 

• Credits earned and completing developmental coursework. ASAP increased the average 
number of credits earned during the first semester by 2. 1 credits and increased the proportion of 
students who completed their developmental coursework by the end of that semester by 15 per- 
centage points. 

• Semester-to-semester retention. ASAP increased the proportion of students who enrolled in 
college during the second semester by 10 percentage points and increased full-time enrollment 
that semester by 2 1 percentage points. 

ASAP’s early effects are larger than the effects of most of the community college programs MDRC 
has studied previously. ASAP’s comprehensive package of financial aid, services, and supports, 
together with its full-time attendance requirement, has resulted in students taking and passing more 
credits than they would have otherwise. Future reports will show whether these effects can be 
sustained — or even grow — as students continue in this comprehensive, three-year program. 
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Preface 


Community colleges across the country confront a clear challenge: too many students arrive on 
campus unprepared, get placed into developmental (or remedial) courses where they stagnate, 
attend only part time (because of work or other responsibilities), and never complete a creden- 
tial, graduate, or transfer to a four-year institution. At the same time, community colleges are 
subject to increasing expectations — and increased scrutiny — about their ability to develop a 
better-educated and credentialed workforce. 

In 2007, The City University of New York (CUNY), with the support of Mayor 
Bloomberg’s Center for Economic Opportunity, launched an ambitious program, Accelerated 
Study in Associate Programs (ASAP), to encourage and support community college students to 
attend school full time and graduate. The program provides a rich array of financial support, 
special seminars and block-scheduled classes, enhanced advising, and other support services for 
three full years. 

This report provides early findings from MDRC’s random assignment evaluation of 
ASAP at three CUNY community colleges, targeted to low-income students who need at least 
one developmental course in reading, writing, or math to build college-level skills. The results 
are very encouraging; for instance, after only one semester, ASAP has increased the proportion 
of students who have completed their developmental education courses by 15 percentage points 
— meaning that 15 more students out of every 100 are ready to take college-level courses. In 
addition, ASAP increased the number of credits students earned during the first semester and 
boosted their rates of full-time enrollment in the first and second semesters. These are some of 
the largest short-term effects we’ve seen in an evaluation of a community college program. 

Recent research by MDRC and others suggests that comprehensive, intensive, and ex- 
tended interventions may be necessary to substantially improve achievement among communi- 
ty college students in the long run. These early results from our evaluation of ASAP provide 
hope that its package of supports and services — and its strong message about full-time 
attendance — are putting students on the right track. Future reports will document whether that 
hope is fulfilled. 

In the meantime, CUNY is using lessons learned from ASAP to infonn the develop- 
ment of a new community college opening in the fall of 2012 — as well as expanding the 
program in its six existing community colleges to serve more than 4,000 students by 2014. 


Gordon L. Berlin 
President 
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Executive Summary 


In recent years, there has been unprecedented national focus on the importance of increasing 
graduation rates for community college students. Many refomis have been tried, but college 
completion rates remain stubbornly low: Only one-third of entering students graduate with a 
degree or certificate within five years. Reforms are often short-term, lasting one or two semes- 
ters, and are designed to address one or only a few barriers to student success. Accelerated 
Study in Associate Programs (ASAP), operated by The City University of New York (CUNY), 
the nation’s largest public urban university system, is an uncommonly multifaceted and long- 
term program aimed at helping community college students stay in school and graduate. 
Launched in 2007 with funding from Mayor Bloomberg’s Center for Economic Opportunity 
(CEO), the program was designed to substantially increase the proportion of students who 
graduate and to help them graduate sooner. 

This report presents very promising early findings from a random assignment study of 
ASAP that is taking place at three CUNY community colleges: Borough of Manhattan Com- 
munity College (BMCC), Kingsborough Community College (KCC), and LaGuardia Commu- 
nity College (LGCC). For the study, ASAP targets low-income students who need one or two 
developmental (remedial) courses to build their math, reading, or writing skills and are willing 
to attend school full time. Compared with regular college services, ASAP increased full-time 
enrollment and the number of credits earned in the first semester of the study, helped students 
complete their developmental requirements that semester, and increased enrollment in the 
second semester. ASAP’s early effects are large, compared with the effects of other community 
college refonns MDRC has studied. 


ASAP Model 

ASAP requires students to attend college full time and provides a rich array of supports and 
incentives for up to three years. Unlike many programs, ASAP is designed to simultaneously 
address multiple potential barriers to student success and to address them over many semesters. 
ASAP’s goal is to graduate at least 50 percent of participating students within three years, far 
exceeding typical graduation rates for low-income developmental students. 

The description below of ASAP’s components is of the program as designed. The eval- 
uation’s research on how ASAP is operated by the colleges is still in progress, but early data 
suggest that ASAP has been generally well implemented, with some variation across the 
components and across the campuses, as is allowed for in the model. 
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• Messages and requirements. ASAP requires students to enroll full time 
every fall and spring semester. The program encourages students to take their 
developmental courses early and to graduate within three years. 

• Course enrollment. Cohorts of students organized by major take three or 
more courses together in a consolidated block schedule during their first year 
in ASAP. For at least the first year of the program, students are required to 
take a non-credit ASAP seminar that covers such topics as goal-setting and 
academic planning. 

• Student services. Students receive comprehensive advisement from their 
ASAP adviser, who has a small caseload (60-80 students). Students are re- 
quired to meet with their adviser at least twice a month. ASAP tutors provide 
general support and conduct review sessions for some courses, and some 
ASAP students are required to attend tutoring frequently. ASAP career and 
employment specialists help with career planning and, if needed, job place- 
ment. Students are required to meet with the career and employment special- 
ist at least once a semester. 

• Financial supports. Any gap between a student’s financial aid and tuition 
and fees is waived, essentially ensuring that all of each student’s tuition and 
fees are covered. Students also receive free monthly MetroCards for use on 
public transportation. The MetroCard is tied to fulfilling certain program re- 
quirements, such as meeting with an adviser or attending tutoring. Students 
also receive free textbooks for their classes (which they must return at the 
end of the semester). 


ASAP Evaluation and Research Sample 

ASAP originally targeted students at CUNY’s six community colleges who were “college- 
ready” when they entered the program (that is, they did not need any developmental course- 
work). CUNY’s internal evaluation of ASAP found promising effects for participating students, 
and CUNY decided to expand the program and commission an external evaluation. For the 
evaluation, ASAP targets exclusively students who need developmental education. 

MDRC, a nonprofit, nonpartisan education and social policy research finn, is using a 
random assignment research design to study the impacts (or effects) of ASAP, compared with 
standard services and courses at the colleges, on students’ academic outcomes over a three-year 
period. This report examines impacts during students’ first and second semesters in the study 
based on student transcripts and CUNY Assessment Tests. 
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For the study, ASAP targets students at BMCC, KCC, and LGCC who met the follow- 
ing eligibility criteria at the point of random assignment: had family income below 200 percent 
of the federal poverty level and/or were eligible for a Pell grant, needed one or two developmen- 
tal courses based on CUNY Assessment Tests, was a new student or a continuing student who 
had earned 12 or fewer credits, was a New York City resident, was willing to attend college full 
time, and was in an ASAP-eligible major (the colleges excluded a few majors that have re- 
quirements that make graduating quickly difficult). Each eligible student who agreed to partici- 
pate was assigned, at random, either to the program group, whose members have the opportuni- 
ty to participate in ASAP, or to the control group, whose members receive the colleges’ 
standard services. Generally, like at most community colleges, the standard services are far less 
intensive than ASAP’s. The colleges randomly assigned students at two points in time: One 
group (or cohort) of students was assigned just before the spring 2010 semester and the other 
just before the fall 2010 semester. 

The 896 students in the sample completed a baseline demographic survey just before 
they were randomly assigned. Roughly two-thirds of the students in the research sample are 
women and most are relatively young (83 percent were 24 years old or younger when they 
entered the study). Reflecting the student body at the three colleges, the study sample is racially 
and ethnically diverse, with no racial or ethnic majority. Virtually all the sample members said 
they planned to obtain a degree beyond an associate’s degree. 


Overview of ASAP’s Early Impacts 

This section summarizes the key impact findings. All of the impacts discussed below are 
statistically significant, meaning that they are unlikely to have arisen by chance. 

• During the first semester, ASAP increased the proportion of students 
who enrolled full time and increased the average number of credits stu- 
dents earned. 

As Table ES.l shows, during the first semester of the follow-up period, 96 percent of the 
program group enrolled in college full time (12 or more credits), compared with 85 percent of the 
control group, yielding an impact of 1 1 percentage points. As noted above, willingness to enroll 
full time was an eligibility requirement for the study. However, this willingness does not always 
translate into actual full-time enrollment; ASAP increased the proportion of students who 
followed through. The impact on full-time enrollment is important, as past research has found 
that part-time enrollment is associated with lower rates of persistence and progress in college. 

Not surprisingly given the increase in full-time enrollment, program group members at- 
tempted more credits during the first semester. They also earned more credits and thus made 
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Evaluation of Accelerated Study in Associate Programs (ASAP) for 
Developmental Education Students 

Table ES.l 


Early Impacts on Selected Academic Outcomes 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

First semester 

Enrolled 2 (%) 

96.4 

94.0 

2.5 * 

Full time 

95.8 

85.2 

10.6 *** 

Part time 

0.6 

8.8 

g j *** 

Total credits attempted 

16.1 

13.9 

2 2 *** 

College-level credits 

10.5 

10.3 

0.2 

Developmental credits 

5.6 

3.6 

2 Q *** 

Total credits earned 

11.4 

9.3 

2 | *** 

College-level credits 

8.5 

7.6 

q g *** 

Developmental credits 

2.9 

1.7 


Completed developmental requirements' 3 (%) 

46.6 

31.9 

14 7 *** 

Second semester 

Enrolled 2 (%) 

90.3 

80.4 

g g *** 

Full time 

80.5 

59.8 

20.6 *** 

Part time 

9.8 

20.6 

-10.8 *** 

Total credits attempted 

11.8 

9.9 

1 g *** 

College-level credits 

10.0 

8.3 

J rj *** 

Developmental credits 

1.8 

1.6 

0.3 

Sample size (total = 896) 

451 

445 



SOURCE: MDRC calculations from CUNY Institutional Research Database (IRDB). 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical 
significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
Estimates are adjusted by site and cohort. 

First semester measures include the main session and intersession. Second semester 
measures do not include intersession data and are subject to change. 

“Developmental credits” are credits associated with developmental reading, writing, and 
math courses; English as a Second Language classes; and a small number of other non- 
college-level courses. CUNY refers to these credits as “equated credits.” 

“Enrollment is based on courses that students are still enrolled in as of the end of the 
add/drop period. Full-time enrollment is defined as enrollment in 12 or more credits. Part- 
time enrollment is defined as enrollment in fewer than 12 credits. 

b Completion of developmental requirements is contingent upon passing CUNY 
Assessment Tests, passing the highest level of developmental education, and/or passing a 
college-level class in each subject. This measure includes students who passed CUNY 
Assessment Tests prior to the first semester. 
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more progress. Program group members earned an average of 11.4 total credits in the first 
semester, 2.1 credits more than the control group — 0.9 college-level credits and 1.1 credits in 
developmental courses. Developmental courses do not provide college-level credits but instead 
provide what are called “equated credits” at CUNY, which count toward part-time/full-time 
status and for tuition and financial aid purposes. Earning these credits, referred to as “develop- 
mental credits” in this report, is an important indicator of progress through developmental 
education. As mentioned above, one of the eligibility criteria for the study was being in need of 
one or two developmental courses. Before students begin classes, they take the CUNY Assess- 
ment Tests, designed to measure basic skills proficiency in reading, writing, and math. Typical- 
ly students at CUNY’s community colleges who fail the tests are not required to take develop- 
mental courses when they start school, but they must complete any developmental courses 
before they can take certain gatekeeper courses, such as freshman English, college algebra, and 
most humanities courses, and before they can graduate or transfer to a CUNY four-year school. 

• ASAP increased the proportion of students who had completed all of 
their developmental requirements by the end of the first semester. 

As Table ES.l shows, 47 percent of the program group had completed their develop- 
mental requirements, compared with 32 percent of the control group. After only one semester, 
ASAP helped a substantial number of students reach this important milestone. 

• ASAP increased the proportion of students who returned to school in 
the second semester. 

All three colleges have “main sessions” (like traditional fall and spring terms) followed 
by shorter “intersessions.” The data for the second semester that were analyzed for this report 
include only the main session at each college (data for the first semester includes both the main 
session and the intersession). As Table ES. 1 shows, 90 percent of the program group members 
enrolled during the main session of the second semester, compared with 80 percent of the 
control group members, a difference of 10 percentage points. The vast majority of the program 
group students enrolled full time: 81 percent, compared with only 60 percent of the control 
group, yielding a substantial impact of 21 percentage points. Since intersession credits count 
toward full-time status, these percentages will change as more data become available. It is 
unlikely, though, that the additional data will change the finding that ASAP positively affected 
second-semester enrollment. During the second semester, the program increased the average 
number of credits that students attempted. (Data on earned credits in the second semester were 
not yet available for the full sample when the analysis for the report was completed.) 

• For the first cohort of sample members, ASAP increased credits earned 
during the second semester and increased the proportion of students 
who enrolled in school in the third semester. 
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An additional semester of follow-up data was available for the first cohort of students in 
the study, assigned just before the spring 2010 semester, which comprises approximately one- 
third of the full sample. Results for the first cohort (not shown) suggest that ASAP is likely to 
continue having positive effects, at least through the start of the third semester. 


Conclusions and What’s Next 

ASAP’s early effects on students’ academic outcomes are very promising, and it is clear at this 
juncture that ASAP’s package of requirements, messages, services, and financial benefits can 
improve short-term outcomes. In fact, ASAP’s effect on the average number of total credits 
students earned during the first semester is larger than the effect of any of the other community 
college programs that MDRC has studied. ASAP’s effect on second-semester enrollment rates 
is the second largest MDRC has found. Based on ASAP’s effects for students to date — those 
in the MDRC random assignment study as well as students in prior entering cohorts — CUNY 
is expanding the program to serve over 4,000 students by 2014, three times its current size. 

Future reports will present findings on longer-term academic outcomes, including two- 
and three-year graduation rates. MDRC’s evaluation will also examine the implementation of 
ASAP and the key differences between ASAP and standard college services and courses. 
Although the evaluation is not designed to definitively determine which components of ASAP 
matter the most — because the entire ASAP package is being compared with services as usual 
— the implementation research will shed light on that issue. Finally, MDRC will also examine 
the costs of ASAP. Given that ASAP is multifaceted and lasts three years, it is likely to be 
more expensive than most other programs studied. It may also, however, generate more 
substantial effects. 
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Introduction 

In recent years, there has been unprecedented national focus on the importance of increasing 
graduation rates for community college students to help improve individuals’ opportunities and 
to build the country’s workforce. Many reforms have been tried, but college completion rates 
remain stubbornly low. Policymakers and colleges continue to seek initiatives that can substan- 
tially improve student success. 

Reforms in community colleges are often short-term, lasting one or two semesters, and 
are designed to address one or a few barriers to student success. This report presents very 
promising early findings from a random assignment study of a program that is uncommonly 
multifaceted and lasts for three years. The City University of New York (CUNY), the nation’s 
largest public urban university system, offers Accelerated Study in Associate Programs (ASAP) 
at all six of its existing community colleges. 1 Designed to help community college students stay 
in school and graduate, ASAP is one of the most ambitious efforts in the country to improve the 
success rates of low-income postsecondary students. Launched in 2007 with funding from 
Mayor Bloomberg’s Center for Economic Opportunity (CEO), the program was designed to 
substantially increase the proportion of students who graduate and to help them graduate sooner. 
The sooner an individual attains a degree, the sooner he or she can reap the economic and other 
benefits of that degree. 

ASAP requires students to attend college full time and provides a rich array of supports 
and incentives for up to three years. Unlike many programs, ASAP is designed to simultaneous- 
ly address multiple potential barriers to student success and to address them over many semes- 
ters. ASAP’s goal is to graduate at least 50 percent of participating students within three years, 
far exceeding typical community college graduation rates. The program model includes the 
following key components: 

• Requirement to attend college full time during the fall and spring semesters 
and encouragement to complete developmental education early and graduate 
within three years 

• Block-scheduled classes with other ASAP students (for the first year of the 
program) 

• ASAP seminar covering such topics as goal-setting, study skills, and aca- 
demic planning (for at least two semesters of the program) 


Yhe ASAP model informed the development of CUNY’s New Community College (NCC), which is 
scheduled to open in fall 2012. 
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Free use of textbooks 


• 

• Comprehensive advisement from an ASAP adviser with a small caseload 

• Tutoring 

• Career services 

• Tuition waiver that covers any gap between a student’s financial aid and tui- 
tion and fees 

• Free MetroCards for use on public transportation 

For the study, ASAP targets low-income students who need one or two developmental 
(remedial) courses based on CUNY Assessment Tests and are willing to attend school full time. 

MDRC, a nonprofit, nonpartisan education and social policy research firm, is conduct- 
ing an evaluation of ASAP, examining its implementation and its effects on students’ academic 
outcomes, including enrollment, credit accumulation, and graduation, over three years. The 
evaluation is taking place at three of CUNY’s six community colleges: Borough of Manhattan 
Community College, Kingsborough Community College, and LaGuardia Community College. 
The study began in 2009 and will continue through 2014. 

This report is the first from the study. The following sections present some background 
on community colleges and infonnation from prior research on initiatives to increase student 
success; a description of the evaluation, including a discussion of the participating colleges and 
the sample members; more detail on the ASAP model; the impacts of ASAP on academic 
outcomes during the first two semesters of the study’s follow-up period; and conclusions and 
next steps. The bottom line? Compared with regular services and classes, ASAP increased full- 
time enrollment and the number of credits students earned in their first semester. It also in- 
creased the proportion of students who returned to college for a second semester. 


Community Colleges and Efforts to Increase Student Success 

Higher education is becoming increasingly important in the labor market; workers with a 
college degree earn substantially more, on average, than those without. For example, data from 
the U.S. Census Bureau show that, in 2010, the average annual earnings for full-time workers 
over the age of 25 with an associate’s degree were $49,275, compared with average annual 
earnings of $40,900 for those who did not advance beyond a high school diploma or General 
Educational Development (GED) certificate. 2 Similarly, an adult with a bachelor’s degree earns 

2 U.S. Census Bureau (2010). 
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about $2.1 million in his or her lifetime, nearly twice as much as an adult with a high school 
diploma or the equivalent. 3 Higher education is also associated with other benefits, such as 
better health and increased civic engagement. 4 

Community colleges play a central role in American higher education. Recent data 
show that the nation’s 1,167 community colleges enroll a total of 12.4 million students, or 44 
percent of the country’s undergraduate population. 5 Owing in part to their relative affordability 
and accessibility, community colleges serve disproportionately more disadvantaged students 
than four-year institutions. For example, low-income, black, and Hispanic college students — 
groups historically underrepresented in higher education — are disproportionately enrolled in 
community colleges and make up a considerable fraction of the student population. 6 Recently, 
the Obama administration has intensified the focus on community colleges, recognizing their 
importance in helping increase the nation’s college graduation rates. 

Unfortunately, completion rates at community colleges are quite low: Only 32 percent 
of entering students graduate with a degree or certificate within five years. 7 Moreover, many 
undergraduates take longer to complete their degrees than is considered “nonnal” — four years 
for a bachelor’s degree and two years for an associate’s. In a survey of students who started at 
public two-year colleges, for example, only 4 percent graduated with an associate’s degree in 
the expected two years. Another 10 percent took three years, and another 9 percent took more 
than three years. 8 The consequences of this increased time to degree are considerable; the longer 
a student remains in school without completing, the more likely it is that he or she will encoun- 
ter issues that will make it necessary to drop courses, take time off, or drop out of college 
altogether. 

While graduation rates are low overall at community colleges, they are even lower for 
the many students who enter college with some developmental needs. Recent data from 
CUNY’s community colleges, for example, show that only 26 percent of students with devel- 
opmental needs graduate after six years, compared with 40 percent of students without any 
remediation needs. 9 Nationwide, nearly 60 percent of community college students enroll in at 
least one developmental reading, writing, or math course. 10 This lack of preparation sets 
students up to struggle academically, increasing the likelihood that they will fail and have to 


3 Day and Newburger (2002). 

4 Baum and Ma (2007); Dee (2004). 

5 American Association of Community Colleges (2011). 

6 Provasnik and Planty (2008). 

7 Horn and Berger (2004). 

8 Berkner, He, and Cataldi (2002). 

9 CUNY Office of Academic Affairs (2011). 

10 Bailey (2009). 
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repeat required courses. Indeed, developmental courses are negatively correlated with student 
success and persistence. 11 

Many non-academic factors can also impede students’ progress through community 
college, including the costs of attending college and a general lack of student support services. 12 
Furthermore, for various reasons, many students attend school part time. For example, while 
most CUNY community college students start out by attending college full time, nearly half 
drop to part-time status at some point during their college careers. 13 This substantially decreases 
their chances of obtaining a postsecondary degree in a timely fashion. Generally, it has been 
established that part-time attendance is negatively associated with persistence. 14 

Many reforms have been implemented to increase rates of college persistence and 
completion for disadvantaged students. MDRC has studied a variety of initiatives over the past 
decade, including financial aid refonns, enhancements to student services, and instructional 
refomis. A variety of approaches have been found to help improve students’ educational 
outcomes, but, for the most part, the effects have been modest. MDRC’s research to date 
suggests that refonns that address one or a few barriers to student success and last one or two 
semesters may not be robust enough to substantially improve long-tenn outcomes. The 
evidence points to the need for multifaceted, longer-lasting interventions and broader institu- 
tional reforms. 15 

CUNY’s ASAP initiative builds upon the modest successes of some earlier programs 
by bringing together multiple strategies into one program and providing services for an extend- 
ed period — three years — to help students stay in school and graduate quickly. It also requires 
full-time enrollment. 

Although ASAP’s rich array of services is not common, other refonns are moving in 
the same direction. For example, the ASAP design infonned the development of a new CUNY 
community college that is scheduled to open in fall 2012. 16 In addition, the Gates Foundation’s 
new Completion by Design initiative provides funds and technical assistance to groups of 
colleges to design new pathways for the entire college experience, from entry to graduation. 17 
When ASAP was launched in 2007, it was at the forefront of a growing effort of bold refonns 
that aim to substantially and meaningfully improve students’ outcomes. 


1 A del man (2004); Attewell, Lavin, Domina, and Levey (2006). 
12 See, for example, Brock and LeBlanc (2005). 

13 Linderman and Kolenovic (2009). 

14 Hoachlander, Sikora, and Horn (2003); Choy (2002). 

15 See, for example. Scrivener and Coghlan (2011). 

16 Linderman and Kolenovic (2012). 

17 See http://www.completionbydesign.org/about-us. 
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The Evaluation 


Background and Key Components 

In 2007, CUNY received funding from Mayor Bloomberg’s Center for Economic Op- 
portunity (CEO) to establish ASAP and launched the program that fall. ASAP originally 
targeted students at CUNY’s six community colleges who were “college-ready” — that is, 
students who were deemed to be fully proficient in reading, writing, and math when the pro- 
gram began. 18 CUNY’s internal evaluation of ASAP found substantial effects on two-year 
graduation rates for the 2007 cohort of students. 19 In 2009, CUNY decided to expand the 
program and commission an external evaluation. For the evaluation, ASAP targets exclusively 
students who need developmental education. 

MDRC is using a random assignment research design to study the impacts (or effects) 
of ASAP, compared with standard services and courses at the colleges, on students’ academic 
outcomes over a three-year period. The impact analysis is examining ASAP’s effects on earned 
credits, progress in developmental education, and degree or certificate attainment. The study is 
also looking at other outcomes, including enrollment, persistence, and transfer to four-year 
institutions. 

MDRC’s evaluation is also exploring how ASAP is implemented at the three colleges 
and how the planned program services compare with the offered services. The study is also 
examining participation in various services for program and control group members and the 
nature of the treatment contrast — that is, the difference between the services received by and 
experiences of program group and control group students. Although the evaluation is not 
designed to definitively determine which components of ASAP matter the most — because the 
entire ASAP package is being compared with services as usual — the implementation research 
should shed light on that issue. 

Finally, MDRC is examining the costs of ASAP. Given that ASAP provides many ser- 
vices and supports for three years, it is likely to be more expensive than most other programs 
studied. It may also, however, generate more substantial effects. 


18 Students’ basic skills proficiency is generally determined by the CUNY Assessment Tests. Twenty-eight 
percent of the fall 2007 cohort had some developmental needs when they were initially recruited for ASAP and 
took their developmental courses during the summer in order to join the program. 

19 Linderman and Kolenovic (2009). CUNY also published a report in early 2012 that describes positive 
effects on three-year graduation rates: Linderman and Kolenovic (2012). 
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Data Sources 


The evaluation of ASAP relies on several data sources. The key analyses discussed in 
this report are based primarily on the following data: 

• Baseline Information Form (BIF). Just before entering the study, all stu- 
dents completed a short survey, called the Baseline Information Form (BIF). 

The BIF collected infomiation on demographic and other background char- 
acteristics. BIF data are used to describe the sample and to examine the simi- 
larity between the two research groups when the study began. Later in the 
study, BIF data may be used to identify students for subgroup analyses. 

• Student Records from CUNY. CUNY is providing to MDRC infonnation 
on students’ academic outcomes from their Institutional Research Database 
(IRDB). These records include transcript data from the three community col- 
leges in the study and the other colleges in the CUNY system, as well as 
from CUNY Assessment Test data. These data are used to determine the 
program’s effect on students’ progress in college and how many develop- 
mental courses sample members needed when they entered the study. 

The report also uses data from the Integrated Postsecondary Education Data System 
(IPEDS), which is housed at the U.S. Department of Education’s National Center for Education 
Statistics, and the institutional fact books from Borough of Manhattan, Kingsborough, and 
LaGuardia Community Colleges to help describe the colleges. 

Future reports will also use data from the National Student Clearinghouse, which pro- 
vides infonnation on enrollment and degree attainment at colleges across the nation; a student 
survey that asked students in the study questions about their participation in student services, 
engagement on campus, study habits, employment, and how they financed college; interviews 
with administrators and staff and observations at the colleges; and various ASAP program 
documents and data. 

The Colleges in the Study 

CUNY is the largest urban public university system in the country, comprising 23 insti- 
tutions, including six community colleges across the five boroughs of New York City. 20 CUNY 
serves more than 540,000 students each year. 21 


20 A seventh community college will open in fall 2012. 
21 See the CUNY website (www.cuny.edu/about.html). 
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ASAP has operated at all six CUNY community colleges since it was launched in 2007. 
Three are participating in the evaluation: Borough of Manhattan Community College (BMCC), 
Kingsborough Community College (KCC) in Brooklyn, and LaGuardia Community College 
(LGCC) in Queens. The colleges were selected by CUNY and MDRC based on their willing- 
ness to participate in an evaluation and work with MDRC to develop and implement random 
assignment, their capacity to reach the desired sample size goals, and the interest of the ASAP 
program administrators in serving additional students. 

Table 1 shows selected characteristics of degree-seeking undergraduates at BMCC, 
KCC, and LGCC in fall 2009. The three colleges are the largest of the six CUNY community 
colleges, enrolling a total of 50,000 students, representing 63 percent of the total enrollment in 
CUNY’s community colleges. The majority of students at the three colleges in fall 2009 
attended full time (about two-thirds at BMCC and LGCC and about three-fourths at KCC). 

As the table shows, the majority of students at the three colleges were women. Across 
the colleges, between 3 1 and 44 percent were younger than 20 years old, between 35 and 39 
percent were 20-24, and between 22 and 32 percent were 25 or older. The student body at the 
three participating colleges, like the study body at other CUNY colleges and like the population 
of New York City more broadly, is racially and ethnically diverse. There is no racial or ethnic 
majority at any of the colleges. The plurality of students is Hispanic at two colleges (BMCC and 
LGCC) and white at one (KCC). 

Just over half of the students at BMCC received a Pell grant, the main federal source of 
need-based financial aid, in the 2008-2009 school year. At the other two colleges, just over one- 
third of students received a Pell grant. 

Finally, as the last row of the table shows, the vast majority (90 percent) of students at 
BMCC were identified by the college as in need of some developmental education. About 
three- fourths of students at KCC and LGCC were in this category. 

The Random Assignment Process and the Sample Members 

ASAP targets students who met the following eligibility criteria when they entered the 

study: 

• Low income: Family income below 200 percent of the federal poverty level 
and/or eligible for a Pell grant 

• In need of one or two developmental courses based on CUNY Assessment 
Tests 

• New student or a continuing student who had earned 12 or fewer credits 
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Evaluation of Accelerated Study in Associate Programs (ASAP) for 
Developmental Education Students 

Table 1 

Selected Characteristics of Degree-Seeking Students at Colleges 
Participating in the ASAP Evaluation 

Early Impacts Report 


Characteristic 

BMCC 

KCC 

LGCC 

Total students 

20,871 

14,217 

14,912 

Enrollment (%) 

Full time 

67.0 

74.3 

63.5 

Part time 

33.0 

25.7 

36.5 

Gender (%) 

Male 

40.7 

44.6 

40.4 

Female 

59.3 

55.4 

59.6 

Age a (%) 

Less than 20 years 

30.8 

43.6 

31.9 

20-24 years 

39.0 

34.5 

36.6 

25 years and over 

30.2 

21.9 

31.5 

Race/ethnicity (%) 

Hispanic 

35.5 

14.7 

35.5 

White 

13.6 

35.3 

13.5 

Black 

31.7 

33.9 

18.3 

Asian or Pacific Islander 

11.5 

11.2 

17.1 

Other b 

7.7 

4.9 

15.5 

Receiving Pell grant c (%) 

52.6 

35.9 

35.9 

In need of developmental education 11 (%) 

89.6 

72.6 

75.5 


SOURCES: U.S. Department of Education, National Center for Education Statistics, Integrated Postsecondary 
Education Data System (IPEDS), unless otherwise noted, and institutional fact books. The latest available 
institutional fact books from BMCC (2009-2010), KCC (2009-2010), and LGCC (2011) community colleges are 
available on the colleges' websites. 

NOTES: BMCC indicates Borough of Manhattan Community College, KCC indicates Kingsborough 
Community College, and LGCC indicates LaGuardia Community College. 

Unless otherwise noted, data are based on undergraduate degree-seeking students from fall 2009. 

Missing values are not included in individual variable distributions. 

Distributions may not add to 100 percent because of rounding. 
a Data on age are based on the entire undergraduate student population. 

b Includes American Indians, Alaska Natives, and nonresident aliens, as tracked by the Integrated 
Postsecondary Education Data System (IPEDS), which does not categorize nonresident aliens by race/ethnicity. 

c Financial aid data are for the 2008-2009 school year and are based on the entire undergraduate student 
population. 

d Data on developmental need are based on institutional fact books. BMCC and KCC data are based on first- 
time freshmen. LGCC data are based on first-time degree-seeking students. 





• New York City resident 

• Willing to attend college full time 

• In an ASAP-eligible major (the colleges excluded a few majors that have re- 
quirements that make graduating quickly difficult) 22 

MDRC worked with staff from the CUNY Office of Academic Affairs and the partici- 
pating colleges to develop procedures to build the research sample. ASAP staff at each college 
invited eligible students to participate in the evaluation through letters, emails, and phone calls. 
Students who attended an intake session on campus, during which staff described the ASAP 
program and evaluation, and agreed to take part in the study completed an Informed Consent 
Form and Baseline Information Form (BIF) containing questions about students’ background 
characteristics. After completing these fonns, each student was assigned, at random, either to 
the program group, whose members have the opportunity to participate in ASAP, or to the 
control group, whose members receive the college’s standard services. As compensation for 
their time, students received a one-week MetroCard for use on public transportation. 23 The 
colleges randomly assigned two groups (or cohorts) of students for the ASAP evaluation: One 
just before the spring 2010 semester and the other just before the fall 2010 semester. (BMCC 
and KCC assigned students before both semesters; LGCC assigned students only before the fall 
2010 semester.) 

Table 2 presents selected characteristics of the 896 individuals in the research sample. 
Because the sample consists of students who met the program eligibility criteria described 
above and agreed to participate in the study, it should not be considered representative of the 
broader body of degree-seeking undergraduates at the three colleges as described in Table 1 . 

The first column of the table shows characteristics for the full sample. The majority of 
participants in the study are women (62 percent). This is slightly higher than the proportion of 
degree-seeking undergraduate women in the student body at each of the colleges. Over half (57 
percent) were younger than 20 when they entered the study, and another one- fourth (26 percent) 
were between 20 and 24. In other words, most sample members were of “traditional” college 
age. The ASAP sample is younger than the overall college populations; this is not surprising 
since ASAP targets new and relatively new students. Only a small minority of the students in 
the study was married or had any children when they entered the study, and most lived with 
their parents. 


22 The excluded majors are: at BMCC, Allied Health Sciences, Pre-Clinical Nursing, Forensic Science, and 
Engineering Science; at KCC, Nursing; and at LGCC, Allied Health Sciences and Engineering Science. 

23 The "unlimited” card covered subway and bus fares for seven days and cost $27. 
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Evaluation of Accelerated Study in Associate Programs (ASAP) for 
Developmental Education Students 

Table 2 

Selected Characteristics of Sample Members at Baseline 


Early Impacts Report 



Full 

Sample 

BMCC 

KCC 

LGCC 

Gender (%) 

Male 

37.9 

41.6 

37.2 

30.6 

Female 

62.1 

58.4 

62.8 

69.4 

Age (%) 

Less than 20 years 

57.1 

58.6 

58.2 

51.8 

20-24 years 

26.0 

26.7 

23.7 

28.8 

25 years and over 

16.9 

14.7 

18.2 

19.4 

Average age (years) 

21.5 

21.1 

21.4 

22.5 

Marital status (%) 

Married 

6.1 

3.7 

6.5 

11.2 

Unmarried 

78.6 

81.3 

76.6 

75.9 

Missing 

15.3 

15.0 

16.9 

12.9 

Lives with parents (%) 

73.7 

75.1 

75.0 

68.1 

Parents pay more than half of expenses (%) 

41.0 

42.9 

40.3 

37.6 

Missing 

18.0 

15.2 

20.3 

20.0 

Race/Ethnicity a (%) 

Hispanic 

43.6 

48.6 

27.8 

60.7 

White 

10.0 

4.1 

19.3 

6.5 

Black 

34.3 

36.0 

39.9 

20.2 

Asian or Pacific Islander 

7.5 

6.9 

8.8 

6.5 

Other 13 

4.6 

4.4 

4.2 

6.0 

Has one or more children (%) 

15.3 

11.8 

17.9 

18.5 

Currently employed (%) 

31.3 

28.0 

31.3 

39.2 

Among those currently employed: 

Number of hours worked per week in current job (%) 

1-10 hours 

8.1 

6.7 

8.9 

9.4 

11-20 hours 

34.0 

41.9 

31.1 

25.0 

21-30 hours 

31.7 

33.3 

32.2 

28.1 

31-40 hours 

24.7 

16.2 

27.8 

34.4 

More than 40 hours 

1.5 

1.9 

0.0 

3.1 

Highest grade completed (%) 

10th grade or lower 

7.3 

7.5 

1A 

6.5 

1 1th grade 

7.8 

8.7 

6.5 

8.2 

12th grade c 

75.9 

75.6 

74.5 

79.4 

Missing 

9.0 

8.2 

11.7 

5.9 


(continued) 
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Table 2 (continued) 



Full 

Sample 

BMCC 

KCC 

LGCC 

Diplomas/degrees earned 11 (%) 

High school diploma 

73.8 

77.3 

71.3 

70.2 

General Educational Development (GED) certificate 

20.8 

19.2 

22.3 

22.0 

Occupational/technical certificate 

5.6 

5.1 

4.8 

8.3 

Other 

1.7 

2.3 

1.0 

1.8 

None 

6.0 

4.3 

7.6 

7.1 

Date of high school graduation/GED receipt (%) 

During the past year 

49.4 

51.4 

48.6 

46.5 

Between one and two years ago 

13.3 

15.5 

11.7 

11.2 

Between two and five years ago 

13.1 

13.0 

12.9 

13.5 

More than five years ago 

13.2 

12.5 

11.7 

17.6 

Has not earned a diploma/GED c 

6.0 

4.5 

7.4 

7.1 

Missing 

5.0 

3.2 

7.7 

4.1 

Student's status 6 (%) 

Incoming freshman 

60.0 

43.1 

72.0 

77.1 

Returning student 

33.5 

55.4 

15.4 

16.5 

Transfer student 

6.5 

1.5 

12.6 

6.5 

Highest degree student plans to attain (%) 

Associate's 

2.8 

2.3 

4.6 

0.6 

Bachelor's 

31.4 

32.2 

28.5 

34.7 

Master's 

41.6 

40.9 

41.7 

43.1 

Professional or doctorate 

17.8 

19.3 

16.9 

16.2 

Beyond an associate's, unspecified 

6.4 

5.3 

8.3 

5.4 

First person in family to attend college (%) 

30.3 

31.7 

26.2 

34.5 

Highest degree/diploma earned by mother (%) 

Not a high school graduate 

19.5 

22.9 

15.4 

19.4 

High school diploma or GED 

21.8 

18.7 

25.5 

21.8 

Some college, did not complete a degree 

16.0 

17.0 

15.1 

15.3 

College degree (AA, BA, MA, PhD) 

18.6 

20.9 

15.7 

18.8 

Missing 

24.1 

20.4 

28.3 

24.7 

Language other than English spoken regularly in home (%) 

44.7 

45.7 

37.0 

57.1 

Sample size 

896 

401 

325 

170 


SOURCE: MDRC calculations using Baseline Information Form (BIF) data. 

NOTES: BMCC indicates Borough of Manhattan Community College, KCC indicates Kingsborough Community 
College, and FGCC indicates FaGuardia Community College. 

Missing values are only included in variable distributions for characteristics with more than 5 percent of the 
sample missing. 

Distributions may not add to 100 percent because of rounding. 

Characteristics shown in italic type are calculated for a proportion of the full sample. 

Respondents who said they are Hispanic and chose a race are included only in the Hispanic category. 
Respondents who said they are not Hispanic and chose more than one race are included in the Other category. 
b Other includes multi-racial, Native American/ Alaska Native, and other race/ethnicities. 
c This includes students who were currently enrolled in high school at study intake. 

^Distributions may not add to 100 percent because categories are not mutually exclusive. 
e Student' s status at the start of first semester in the study. 


11 







Reflecting the student body at the three colleges, the study sample is racially and ethni- 
cally diverse, with no racial or ethnic majority: 44 percent of sample members identified 
themselves as Hispanic, 10 percent as white, 34 percent as black, and 8 percent as Asian or 
Pacific Islander. Shown on the second page of the table, a total of 60 percent of the sample 
members were incoming freshmen at the start of the study. Virtually all of the students (97 
percent) reported that they planned to attain a degree beyond an associate’s degree and over half 
planned to obtain a graduate degree. 

Although the evaluation’s impact analysis pools the sample members from the three 
colleges, Table 2 shows characteristics separately for BMCC, KCC, and LGCC for informa- 
tional purposes. There are some differences across the three colleges. For example, the sample 
at LGCC is slightly older, somewhat more likely to be married, and somewhat less likely to live 
with their parents. 

Appendix Table 1 presents characteristics for the program group and control group stu- 
dents separately. An asterisk indicates that the percentage of program group members with that 
characteristic is statistically significantly different from the percentage of control group mem- 
bers with that characteristic. As the table shows, the background characteristics of program and 
control group students were very similar at the start of the study, suggesting that the random 
assignment process succeeded in creating two similar groups of students. 

As mentioned above, one of the eligibility criteria for the evaluation was being in need 
of one or two developmental courses based on CUNY Assessment Test scores. Students are 
required to take the CUNY Assessment tests, which are designed to assess basic skills profi- 
ciency in reading, writing, and math, before they begin classes. 24 Typically students at CUNY’s 
community colleges who fail the tests are not required to take developmental courses when they 
start school, but they must complete any developmental courses before they can take certain 
gatekeeper courses, such as freshman English, college algebra, and most humanities courses, 
and before they can graduate or transfer to a CUNY four-year school. 

Table 3 presents the basic skills proficiency levels for the sample members at the point 
they entered the study. 25 The vast majority of students (86 percent) needed one or two develop- 
mental courses, reflecting the eligibility criterion. As the table shows, however, some sample 
members (6 percent) were actually proficient in all subjects — that is, they did not need any 
developmental courses — and some students (8 percent) needed three or more developmental 


24 The colleges also accept scores above a certain threshold on New York Regents tests, the SAT, and the 
ACT as evidence of college-level skills. 

25 Skills test data were not available for all of the sample members; the calculations in Table 3 are based on 
the 789 students (out of 896 students in the study) for whom all relevant placement data were available. 
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Evaluation of Accelerated Study in Associate Programs (ASAP) for 
Developmental Education Students 

Table 3 

Developmental Courses Needed at Baseline 


Early Impacts Report 


Outcome (%) 

Full 

Sample 

Number of developmental courses needed 

None 

6.1 

One 

37.1 

Two 

49.2 

Three or more 

7.6 

Subject of developmental need 

English 

18.3 

Math 

49.6 

English and math 

26.1 

Sample size 

789 


SOURCE: MDRC calculations from CUNY Institutional Research Database (IRDB). 

NOTES: Students without pre-random assignment CUNY Assessment Test data in all subject areas are excluded 
from this table. 

Rounding may cause slight discrepancies in sums and differences. 

Estimates are adjusted by site and cohort. 


courses. While the inclusion in the sample of students who did not meet one of the eligibility 
criteria is not ideal, it would raise more concerns if the proportion of technically ineligible 
sample members differed by research group because that would suggest that the program and 
control groups were not comparable. In fact, the percentage who did not meet the criterion is 
similar in the program and control groups (not shown in the table). 

Table 3 also shows the basic skill area(s) in which students needed to remediate. Similar 
to developmental students across the nation, most sample members needed to build their math 
skills: 50 percent of the students in the sample needed just developmental math and another 26 
percent needed developmental math and English (reading or writing). A total of 18 percent 
needed just developmental English. 26 


26 These numbers do not sum to 100 percent because, as shown in the table, 6 percent of the sample did not 
need any remediation. 
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The ASAP Model in More Detail 

Before turning to a discussion of ASAP’s early impacts on students’ academic outcomes, this 
section provides additional information on the program model. It is important to note that the 
description below is of ASAP as designed. The evaluation’s implementation research is still in 
progress, but early data suggest that ASAP has been generally well implemented, with some 
variation across the components and across the campuses, as is allowed for in the model. Future 
reports will describe findings from the full implementation research effort. 

It is important to emphasize that ASAP requires participating students to attend school 
full time. This is not only an eligibility requirement for entering the ASAP program (and the 
evaluation of ASAP), but students must continue to enroll full time in order to remain in ASAP 
from semester to semester. One novel aspect of ASAP is that not only does it require full-time 
attendance, but it also provides students with a full range of supports (described below) intended 
to make such attendance possible. For example, ASAP covers any tuition and fees not covered 
by financial aid, allowing students to attend full time, rather than part time, without paying for 
any additional courses. This is important, since full-time enrollment may enable students to 
complete community college in a more timely fashion. 

ASAP bundles together several strategies and takes a holistic approach, addressing mul- 
tiple barriers to success. Below is a detailed description of the key components of the ASAP 
program model. Most components are provided for three years (except where noted). It is 
notable that while ASAP provides a broad array of supports, it also requires and expects a lot 
from participating students. 

Messages 

• Enroll full time. ASAP requires students to enroll full time in each fall and 
spring semester they are in the program. 

• Take developmental courses early. ASAP encourages students to take any 
developmental courses they need to early in their time in college. 

• Graduate quickly. Finally, ASAP encourages students to graduate with an 
associate’s degree within three years. 

Course Enrollment 

• Block-scheduled classes. Groups of students organized by major take three 
to five courses together in a consolidated block schedule during their first 
year in the ASAP program. The block includes the ASAP seminar, described 
next, and typically a developmental education course. A consolidated course 
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schedule feature, in which students can take all their classes in the morning 
or afternoon, is available throughout students’ time in the program, but stu- 
dents can take classes outside the consolidated schedule. 

• ASAP seminar. For at least two semesters, ASAP students are required to 
take a non-credit seminar that covers such topics as goal-setting, study skills, 
academic planning, and career-related issues. 

Student Services 

• Comprehensive advisement. Students are assigned to an ASAP adviser 
with a small caseload: 60-80 students. (Typical community college advisers 
can be responsible for 1,000 students or more.) ASAP advisers are expected 
to provide comprehensive academic, social, and interpersonal support. Stu- 
dents are required to meet with their advisers at least twice a month, allowing 
issues to be raised and addressed in a timely manner and ensuring that any 
necessary referrals to tutoring, psychological counseling, or other services 
can be made. 

• Enhanced tutoring. ASAP tutors support students both inside and outside 
the classroom. Tutors provide general support and conduct regular review 
sessions for especially challenging courses. Some ASAP students are re- 
quired to attend tutoring frequently, such as those taking developmental 
courses or those on academic probation. 

• Career services. ASAP career and employment specialists provide help with 
career planning, job skills, interviewing, and, if needed, job placement. They 
also coordinate career fairs, guest speakers, and visits to selected worksites. 
Students are required to meet with a career and employment specialist at 
least once per semester. 

Financial Supports 

• Tuition waiver. Any gap between a student’s financial aid and tuition and 
fees is waived. (ASAP requires all students to complete a Free Application 
for Federal Student Aid [FAFSA].) 

• Free MetroCards. Students receive free monthly MetroCards for use on 
public transportation. The MetroCard is tied to fulfilling certain program re- 
quirements, such as meeting with an adviser or attending tutoring. 
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• Free use of textbooks. ASAP students receive free textbooks for all their 
classes. (Students must return the textbooks at the end of each semester.) 

The program also includes periodic social events for participating students and an 
ASAP-wide student leadership program in which groups participate in leadership-building 
workshops. The colleges sometimes have a social work intern from the Hunter College School 
of Social Work, who is available to meet with students about personal issues and provide 
counseling. 

As noted above, the control group in the evaluation has access to the colleges’ standard 
services and courses. Generally, like at most community colleges, the standard college services 
are far less intensive than ASAP’s. Future reports will describe the key differences between 
ASAP and the services available to the control group students. 


Early Impacts 

This section of the report presents findings on ASAP’s effects on students’ academic progress 
during the first and second semesters of the program. The focus of these early findings is on 
whether the opportunity to participate in ASAP improved students’ progress through develop- 
mental education and enabled students to progress towards earning a degree more quickly than 
they would have had they been offered their colleges’ “regular” programs and services. 

As described in more detail in Box 1, the program’s impact or effect is estimated by 
comparing the outcomes of all students who were randomly assigned to the program group with 
the outcomes of all students who were randomly assigned to the control group. 27 Random 
assignment results in two groups of students that are similar at the outset of the study, both with 
respect to their observable characteristics (for example, gender, age, and race) as well as 
unobservable characteristics (for example, tenacity, ability, and motivation). As a result, 
subsequent substantial differences in outcomes between the two groups can confidently be 
attributed to the opportunity to participate in ASAP, rather than to preexisting differences 
between the two groups. 

All analyses reflect the effect of the opportunity to participate in ASAP, which is not 
necessarily the same as the effect of participation in ASAP. 28 Some individuals assigned to the 


27 An ordinary least squares regression model is used to obtain estimates. All models include interactions 
of college and cohort as covariates, reflecting the fact that random assignment occurred within college and 
cohort. 

28 That is, the analyses are intent-to-treat (ITT), as described by Bloom (1984) and Angrist, Imbens, and 
Rubin (1996). 
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Box 1 


How to Read the Impact Tables in This Report 

Most tables in this report use the format illustrated in the abbreviated table below, which displays 
some hypothetical transcript data for the program and control groups. The first row shows that 
program group students earned an average of 9.1 credits and control group students earned an 
average of 6.1 credits. 

The “Difference” column in the table shows the observed difference between the two research groups 
on the outcome — that is, the estimated average impact of the opportunity to participate in the pro- 
gram. For example, the estimated average impact on credits earned can be calculated by subtracting 
6.1 from 9.1, yielding an impact estimate of 3.0 credits earned. 

Differences marked with one asterisk or more are considered statistically significant, meaning that 
there is a high probability that the opportunity to participate in the program had an impact on that 
outcome measure. Differences that have no asterisk indicate that the opportunity to participate in the 
program did not have a discernible effect on that outcome. The number of asterisks indicates the 
probability that an impact at least as large as the one observed in the study would have occurred even if 
the true average impact had been zero. One asterisk corresponds to a 10 percent probability; two 
asterisks, a 5 percent probability; and three asterisks, a 1 percent probability. The more asterisks on a 
positive difference, the more likely the opportunity to participate in the program had a true positive 
average impact on the outcome. The impact in the table excerpt below has three asterisks indicating 
that the impact is statistically significant at the 1 percent level — meaning that there is only a 1 percent 
chance of observing an estimated average impact this large (or larger) if the opportunity to participate 
in the program actually had no average effect on credits earned. In other words, there is a 99 percent 
level of confidence that the opportunity to participate in the program had a positive impact on the 
average number of credits earned. 

Also shown in the table is the standard error of the impact estimate. The standard error is a measure of 
uncertainty or variability around the impact estimate. For those familiar with political polling, the 
standard error is used to calculate the margin of error. As an example, when pollsters state that “presi- 
dential candidate A has a 3 percentage point lead over presidential candidate B, with a margin of error 
of ± 2 percentage points,” they use the standard error to determine the margin of error. Conventionally, 
such a statement implies that pollsters are 95 percent confident that candidate A’s “true” lead is be- 
tween 1 and 5 percentage points (3 ± 2), with their best estimate being 3 percentage points. A useful 
rule of thumb is that the margin of error is usually calculated as 1 .96 x standard error (for a 95 percent 
confidence interval). In the example below, the margin of error is 1.6 (1.96 x standard error = 1.96 x 
0.8). Thus there is a 95 percent chance that the “true” average impact on credits earned lies between 
1.4 and 4.6, calculated as 3.0 ± (1.96 x 0.8). 



Program 

Control 

Difference 

Standard 

Outcome 

Group 

Group 

(Impact) 

Error 

Credits earned 

9.1 

6.1 

3 o *** 

0.8 
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program group did not participate in any part of ASAP; however, they are still included in 
the program group for the analyses in order to ensure a comparison of two groups that were 
similar at the outset of the study. Notably, program participation was very high — 95 percent 
of program group students were identified by the colleges as participating in ASAP during 
the first semester (not shown in tables); consequently, at the end of the first semester of 
study, the effect of the opportunity to participate in ASAP is quite similar to the effect of 
participating in ASAP. 29 

As Box 2 describes in more detail, at two of the three participating colleges (KCC and 
LGCC), the fall and spring semesters are each comprised of a longer “main session” followed 
by a shorter trailing “intersession.” The third college (BMCC) follows a more traditional 
semester arrangement, with a fall semester, a short winter intersession, a spring semester, and 
summer sessions. In order to discuss comparable time periods across colleges, the analyses in 
this report combine the fall and winter semesters at BMCC into “fall” and combine the spring 
and summer into “spring.” In the report, BMCC’s fall and spring semesters are referred to as 
main sessions, and the winter and summer as intersessions. 

Main Findings 

In the short term, students offered the opportunity to participate in ASAP made greater 
academic progress than students offered the colleges’ usual services. Specifically: 

• ASAP had a positive effect on full-time enrollment. During the first se- 
mester of study, 96 percent of the program group students enrolled full time, 
compared with 85 percent of the control group. The 1 1 -percentage-point dif- 
ference is ASAP’s estimated impact, or value-added. 

• ASAP had a positive effect on credits earned. During the first semester, 
students randomly assigned to participate in ASAP earned an average of 1 1 .4 
total credits whereas control group students earned an average of 9.3 total 
credits. The 2.1 credit impact (0.9 college-level credits and 1.1 credits in de- 
velopmental courses) represents an increase of 22 percent over the control 
group base. 

• ASAP had a positive effect on completing developmental requirements. 

Forty-seven percent of the program group had completed all their develop- 


~ 9 That is, the ITT is similar to the treatment on the treated (TOT). Since there were no control group 
“cross-overs” (that is, control group members who ended up participating in ASAP), the TOT is the same as 
the local average treatment effect (LATE), as described by Imbens and Angrist ( 1 994). 
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Box 2 


Timing of Academic Semesters 

Most of the analyses in this report combine outcomes for the study’s two cohorts of students, looking at 
outcomes relative to when students entered the study. For the first cohort, the first semester of the 
follow-up period refers to spring 2010, the second semester refers to fall 2010, and so on. For the second 
cohort, the first semester refers to fall 2010, the second semester refers to spring 2011, and so on. Some 
outcomes in this report are presented only for the first cohort because more follow-up data are available 
for those students: The first cohort has data available through enrollment in the third semester, whereas 
the second cohort has data available only through enrollment in the second semester. 

At two of the three colleges in the evaluation — KCC and LGCC — the academic calendar includes a 
fall semester and a spring semester. Both fall and spring are comprised of two sessions: a 12-week 
session followed by a 6-week session. In this report, the longer first part of the semester is referred to as 
the “main session,” and the shorter second part is referred to as the “intersession.” The 12-week sessions 
tend to have higher enrollment rates than the 6-week sessions, but, unlike colleges whose winter and 
summer courses have very low enrollment rates, these colleges’ intersessions enroll a fairly large 
percentage of students. 

BMCC follows a more traditional semester arrangement, with a 16-week fall semester, a 3 -week winter 
intersession, a 16-week spring semester, and two 6-week summer sessions. At BMCC, the fall and 
spring semesters have far higher enrollment rates than winter and summer. In order to discuss compara- 
ble time periods across colleges, the analyses in this report combine the fall and winter semesters at 
BMCC into “fall” and combine the spring and both summer sessions into “spring.” (If a student attend- 
ed any classes at another CUNY college, their semesters are handled the same way.) In the report, 
BMCC’s fall and spring semesters are referred to as main sessions, and the winter and summer as 
intersessions. 


The table below shows how the semesters are defined in the report for each cohort of sample members. 




2010 

2011 


Spring Semester 
Spring Slimmer 
Session Session 

Fall Semester 
Winter 

Fall Session Session 

Spring Semester 
Spring Slimmer 
Session Session 

Spring '10 
Cohort 

First semester 

Main session Intersession 

Second semester 
Main session Intersession 

Third semester 
Main session Intersession 

Fall '10 
Cohort 


First semester 

Main session Intersession 

Second semester 
Main session Intersession 


NOTE: *KCC refers to the longer session as session A and to the shorter trailing session as session B. 
LGCC refers to the longer session as session 1 and to the shorter trailing session as session II. 
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mental requirements by the end of the first semester, compared with 32 per- 
cent of the control group, yielding an impact of 15 percentage points. 

• ASAP had a positive effect on retention. Compared with control group 
students, program group students were around 10 percentage points more 
likely to enroll in any class during the second semester of study and 2 1 per- 
centage points more likely to enroll full time. 

In general, ASAP has had a positive effect on students’ short-term academic progress. 
These early promising results are described in more detail below. 

Detailed Findings 

During the first semester of study, the ASAP program had a positive effect on students’ 
academic progress with respect to credit accumulation. The details of how these positive effects 
were obtained are unpacked below. The focus is on the programs’ effect on full-time enrollment 
rates, average credits attempted, and credit pass rates, as these represent three distinct (although 
not mutually exclusive) mechanisms through which students can earn more credits and thus 
progress more quickly towards a degree. 

First Semester. Table 4 presents several indicators of academic progress during the 
first semester of study, including enrollment rates, credit attempts, credits earned, credit pass 
rates, and Grade Point Average (GPA). One of the more striking findings in Table 4 is that, 
compared with control group students, program group students were 10.6 percentage points 
more likely to enroll full time during the first semester of study. (In this analysis, the first 
semester includes the “main session” and the “intersession,” as described in Box 2.) 30 Full-time 
enrollment is defined as enrolling in 12 or more credits. Importantly, willingness to enroll full 
time was an eligibility requirement for joining the study. That is, both program group students 
and control group students indicated that they were willing to enroll full time just before they 
were randomly assigned. However, as Table 4 illustrates, willingness to enroll full time does not 
always translate into actual full-time enrollment. The opportunity to participate in ASAP 
resulted in an additional 10.6 percentage points of students acting upon their reported willing- 
ness to enroll full time. 

ASAP’s impact on full-time enrollment is notable because of both its magnitude and its 
implications. In tenns of magnitude, a 10.6 percentage point impact on a base of 85.2 percent is 
remarkable since the largest this impact could be is 14.8 percentage points (had ASAP induced 


°KCC and LGCC sum credits from the main session and intersession to determine full-time status. 
BMCC does not count credits in winter and summer as part of fall and spring; to allow for cross-college 
analysis, the analysis presented in this report does. 
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Evaluation of Accelerated Study in Associate Programs (ASAP) for 
Developmental Education Students 

Table 4 

Academic Outcomes: First Semester 


Early Impacts Report 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Enrolled 3 (%) 

96.4 

94.0 

2.5 * 

1.4 

Full time 

95.8 

85.2 

10.6 *** 

1.9 

Part time 

0.6 

oo 
o o 

_g i *** 

1.4 

Total credits attempted 

16.1 

13.9 

2 2 *** 

0.3 

College-level credits 

10.5 

10.3 

0.2 

0.3 

Developmental credits 

5.6 

3.6 

2 o *** 

0.2 

Total credits earned 

11.4 

9.3 

2 \ *** 

0.4 

College-level credits 

8.5 

7.6 

0 9 *** 

0.3 

Developmental credits 

2.9 

1.7 

l 1 *** 

0.2 

Credit pass rate (%>) 

70.6 

66.9 



College-level credits 

80.8 

73.4 



Developmental credits 

51.4 

48.4 



Term GPA (%) 





2.0 to 4.0 

68.8 

60.2 

8.6 *** 

3.2 

0.0 to 1.9 

26.1 

28.4 

-2.3 

3.0 

No GPA b 

5.1 

11.4 

-6.3 *** 

1.8 

Completed developmental requirements c (%) 

46.6 

31.9 

14 7 *** 

3.2 

Sample size (total = 896) 

451 

445 




SOURCE: MDRC calculations from CUNY Institutional Research Database (IRDB). 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by site and cohort. 

Italic type indicates nonexperimental data. Significance tests are not calculated for nonexperimental data. 

First semester measures include the main session and intersession. 

"Developmental credits” are credits associated with developmental reading, writing, and math courses; English 
as a Second Language classes; and a small number of other non-college-level courses. CUNY refers to these 
credits as "equated credits.” 

a Enrollment is based on courses that students are still enrolled in as of the end of the add/drop period. Full-time 
enrollment is defined as enrollment in 12 or more credits. Part-time enrollment is defined as enrollment in fewer 
than 12 credits. 

b The "No GPA" category includes students who did not enroll and students who took only developmental 
courses, which are not included in GPA calculations. 

c Completion of developmental requirements is contingent upon passing CUNY Assessment Tests, passing the 
highest level of developmental education, and/or passing a college-level class in each subject. This measure 
includes students who passed CUNY Assessment Tests prior to the first semester. 
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every single program group student to enroll full time). Essentially, ASAP induced seven out of 
every ten students who otherwise would either (a) not have enrolled or (b) would only have 
enrolled part time to instead enroll full time. 

In terms of implications, it is fairly well-established that part-time attendance is a “risk 
factor” for community college students, negatively associated with persistence. ASAP’s 
positive effect on full-time enrollment demonstrates that, given the right intervention, a subset 
of students who otherwise would enroll part time will enroll full time. While this study is not 
designed to disentangle with certainty which component(s) of ASAP caused this impact, 
something about ASAP’s combination of requirements and supports successfully increased full- 
time enrollment rates. The impact on full-time enrollment is likely the collective result of 
requiring students to enroll full time in order to remain in a desirable program and providing 
multiple sets of supports (financial, counseling, etc.) to enable students to meet this requirement. 

Related to ASAP’s positive effect on full-time enrollment, the program positively af- 
fected the average number of credits students attempted during the first semester. 31 On average, 
program group students attempted 2.2 more total credits than their control group counterparts. 
This is critical because a program that increases the number of credits students attempt can also 
impact the number of credits students earn even if credit pass rates are unaffected. Notably, 
during the first semester, ASAP primarily impacted the number of “developmental credits” 
students attempted; the program did not have discernible effects on college-level credits 
attempted. Developmental courses do not provide college-level credits but instead provide what 
are called “equated credits” at CUNY, which count towards part-time/Ml-time status and for 
tuition and financial aid purposes. Earning these credits, referred to as “developmental credits” 
in this report, is an important indicator of progress through developmental education. 

With respect to credit accumulation, ASAP had a positive impact on total, college-level, 
and developmental credits earned during the first semester of study. Although ASAP did not 
have a discernible effect on the average number of college-level credits students attempted, 
program group students nonetheless earned 0.9 more college-level credits than their control 
group counterparts. This occurred largely as a result of program group students’ pass rate for 
college-level credits being 81 percent, whereas control group students’ pass rate for college- 
level credits was 74 percent. 

With respect to developmental credits, here program group students attempted 2.0 more 
credits than control group students, and this translated into program group students earning an 
additional 1.1 developmental credits. Notably, pass rates on developmental credits were fairly 


3l The program’s impact on full-time enrollment and credits attempted are highly related since full-time 
enrollment represents reaching or exceeding a certain threshold of credits attempted (12 credits). 
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similar between the two groups, hovering around 50 percent. ASAP’s 1.1 -credit impact on 
developmental credits earned represents a 65 percent increase over the control groups’ baseline 
average developmental credits earned (1.7). 32 This impact is important since developmental 
education is a stumbling block for so many students. That said, there is still much room for 
improvement, since even program group students only earned one out of every two develop- 
mental credits they attempted. 

Summing ASAP’s effect on college-level and developmental credits yields the result 
that program group students earned an average of 2.1 more total credits than their control group 
counterparts during the first semester of study. This represents a 22 percent increase over the 
control group’s baseline average of total credits earned (9.3 credits). 33 To provide some context 
for interpreting this effect, most courses that students in this study attempted were worth three 
or four credits. 

Finally, another way to consider ASAP’s impact on progress through developmental 
education is to examine completion of their developmental coursework. By the end of the first 
semester of study, 46.6 percent of program group students had completed their developmental 
course requirements compared with 31.9 percent of control group students. The 14.7 percentage 
point impact represents ASAP’s value-added over the colleges’ usual services, with respect to 
helping students achieve this milestone after one semester. This 14.7 percentage point impact 
represents a 46 percent increase over the control level. 

First Semester Separated by “Session.” The previous section described the effect of 
ASAP on academic outcomes during the first semester after students entered the study, includ- 
ing the “main session” and the “intersession,” as described above and in Box 2. For a more 
detailed examination of academic progress, Table 5 breaks the first semester into its two 
component sessions. 

During the main session of the first semester, program group students attempted and 
earned more credits than their control group counterparts. In the main session, ASAP’s positive 
effect on total credits attempted was a result of students attempting more developmental credits 
— program and control group students attempted the same number of college-level credits. With 


32 ln effect size units, this impact is 0.38. Effect size is calculated by dividing the impact estimate (1.1) by 
the standard deviation of the outcome (2.98). Since “developmental credits earned” has a zero-inflated 
distribution, it does not have the same properties as a normally distributed outcome. As a result, caution should 
be used when considering the impact in effect size units. 

33 ln effect size units, this impact is 0.33. Effect size is calculated by dividing the impact estimate (2.1) by 
the standard deviation of the outcome (6.19). Since “total credits earned” has a zero-inflated distribution, it 
does not have the same properties as a normally-distributed outcome. As a result, caution should be used when 
considering the impact in effect size units. 
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Evaluation of Accelerated Study in Associate Programs (ASAP) for 
Developmental Education Students 

Table 5 

Academic Outcomes: First Semester, by Session 


Early Impacts Report 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Main session 





Enrolled 3 (%) 

96.4 

93.7 

2.7 * 

1.4 

Total credits attempted 

13.0 

12.1 

0 9 *** 

0.2 

College-level credits 

9.0 

8.9 

0.1 

0.2 

Developmental credits 

4.0 

3.2 

Q g *** 

0.2 

Total credits earned 

9.4 

7.9 

1 5 *** 

0.3 

College-level credits 

7.2 

6.4 

0 g *** 

0.3 

Developmental credits 

2.2 

1.5 

q 7 *** 

0.2 

Credit pass rate (%) 

72.5 

65.7 



College-level credits 

80.2 

72.0 



Developmental credits 

55.3 

48.1 



Intersession 





Enrolled 3 (%) 

58.4 

37.4 

21.0 *** 

2.8 

Total credits attempted 

3.1 

1.8 

1 3 *** 

0.2 

College-level credits 

1.5 

1.4 

0.1 

0.1 

Developmental credits 

1.6 

0.5 

1 1 *** 

0.1 

Total credits earned 

2.0 

1.4 

0.6 *** 

0.2 

College-level credits 

1.3 

1.1 

0.1 

0.1 

Developmental credits 

0.7 

0.2 

0 5 *** 

0.1 

Credit pass rate (%) 

63.3 

74.8 



College-level credits 

84.2 

82.9 



Developmental credits 

43.4 

50.0 



Sample size (total = 896) 

451 

445 




SOURCE: MDRC calculations from CUNY Institutional Research Database (IRDB). 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by site and cohort. 

Italic type indicates nonexperimental data. Significance tests are not calculated for nonexperimental data. 
“Developmental credits” are credits associated with developmental reading, writing, and math courses; English 
as a Second Language classes; and a small number of other non-college-level courses. CUNY refers to these 
credits as “equated credits.” 

a Enrollment is based on courses that students are still enrolled in as of the end of the add/drop period. 
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respect to total credits earned, ASAP students earned an average of 1.5 more credits than their 
control group counterparts. This impact was spread about equally between college-level and 
developmental credits. The impact on college-level credits earned is a result of higher pass rates 
on those credits (80 percent for the program group compared with 72 percent for the control 
group). The impact on developmental credits earned is the result of both program group students 
attempting more developmental credits and passing their attempted credits at a higher rate. 

During the intersession of the first semester, 58.4 percent of program group students en- 
rolled in one or more course, compared with 37.4 percent of control group students. The 
opportunity to participate in the ASAP program had a 2 1 percentage point impact on enrollment 
in the intersession. (Early implementation research suggests that ASAP advisers often encour- 
age students to take courses during the intersessions.) In large part as a result of this increased 
enrollment, program group students attempted 1.3 more credits than their control group coun- 
terparts. 34 Much like during the main session, most of the impact on credits attempted during the 
intersession was a result of program group students attempting more developmental credits (as 
opposed to college-level credits). During the intersession, the program had a 0.6 impact on total 
credits earned, nearly all of which resulted from students taking and earning more credits in 
developmental courses. Pass rates for developmental credits during the intersession were low 
for program and control group students, but since program group students attempted more 
credits, they earned more credits than their control group counterparts. 

ASAP had a positive effect on credit accumulation during intersessions, which are often 
considered optional or secondary at community colleges. Interestingly, MDRC’s experimental 
evaluations of learning communities at KCC and performance-based scholarships at two 
community colleges in New York City and two in Louisiana provide additional examples of 
programs that made a difference during intersessions. 35 Moreover, broader research on corre- 
lates of academic success shows an association between summer credit accumulation and 
students’ probability of completion. 36 Improving enrollment rates during intersessions is almost 
certain to speed up overall progress and may even bridge the gap between main sessions, 
potentially reducing the dropout rate. Since enrollment rates are relatively low during summer 
and winter intersessions, they may be an overlooked area for improving success rates at com- 
munity colleges. 


34 Compared with the control group, an additional 21 percentage points of the program group enrolled. If 
these additional 21 percentage points of students attempted 4.9 credits (the average number of credits that an 
enrolled control group student attempted, calculated as 1.8 / [37.4 / 100]), then they alone would account for an 
impact of nearly a full credit attempted (4.9 x 0.21). 

35 Sommo, Mayer, Rudd, and Cullinan (forthcoming); Richburg-Hayes, Sommo, and Welbeck (2011); 
Richburg-Hayes et al. (2009). 

36 Adelman (2006). 
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Recall that across the entire first semester (including the main session and the interses- 
sion), program group students earned an average of 2.1 more total credits than their control 
group counterparts. This impact was spread across the main session and the intersession, with 
71 percent of the impact (1. 5/2.1) occurring during the main session and 29 percent occurring 
during the intersession (0.6/2. 1). 

Second Semester. Table 6 presents second semester results for enrollment and credits 
attempted during the main session of the second semester (intersession data were not yet 
available for the full sample when the analysis for this report was done). ASAP’s positive effect 
continues and grows during the second semester. In the main session of the second semester, 
90.3 percent of program group students enrolled, whereas 80.4 percent of control group students 
enrolled. The 9.8 percentage point difference cut the percentage of non-enrollees in half (from 
19.6 to 9.7). Moreover, during the main session of the second semester, 80.5 percent of program 
students had already enrolled in enough credits to be deemed full time, whereas only 59.8 
percent of control group students had done so. This represents an impact of 20.6 percentage 
points on full-time enrollment. Since intersession credits attempted count towards students’ full- 
time status, these percentages will change as more data become available, but the general 
finding that ASAP is having a positive effect on students’ likelihood of enrolling during the 
second semester of study will almost undoubtedly hold up. 

Much like the first semester findings, the program increased the average number of 
credits students attempted in the second semester. During the main session, ASAP students, on 
average, attempted an additional 1.9 total credits compared with their control group counter- 
parts. Without the ASAP program, control group students attempted an average of 9.9 credits. 
Program group students attempted 11.8 credits, on average. Unlike the first semester findings, 
the second semester impact on credits attempted is primarily a result of students’ attempting 
more college-level credits — this is likely because almost half of program students had now 
completed their developmental course sequence. For the full sample, data are not yet available 
with respect to credits earned during the second semester of study; that information will be 
presented in future reports. To give a hint at what may be to come, the next section of this report 
briefly presents the program’s effects for the first cohort of students only, for which an addition- 
al semester of follow-up data is currently available. 

First Cohort Findings. Tables 4 through 6 presented the short-term effects of ASAP 
for the full study sample of 896 students. Table 7 presents selected findings for an additional 
semester of follow-up for the first of two cohorts participating in the ASAP study. The first 
cohort, whose first semester in the program was spring 2010, represents approximately one- 
third of the full sample (n = 327) and includes students from two of the studies’ three colleges 
(BMCC and KCC). Table 7 provides an indication of ASAP’s effectiveness after two complete 
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Evaluation of Accelerated Study in Associate Programs (ASAP) for 
Developmental Education Students 

Table 6 

Academic Outcomes: Second Semester 


Early Impacts Report 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Main session 





Enrolled 3 (%) 

90.3 

80.4 

g g *** 

2.3 

Full time 

80.5 

59.8 

20.6 *** 

2.9 

Part time 

9.8 

20.6 

-10.8 *** 

2.3 

Total credits attempted 

11.8 

9.9 

l g *** 

0.3 

College-level credits 

10.0 

8.3 

1 j *** 

0.3 

Developmental credits 

1.8 

1.6 

0.3 

0.2 

Sample size (total = 896) 

451 

445 




SOURCE: MDRC calculations from CUNY Institutional Research Database (IRDB). 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by site and cohort. 

Second semester measures do not include intersession data and are subject to change. 

“Developmental credits” are credits associated with developmental reading, writing, and math courses; English 
as a Second Language classes; and a small number of other non-college-level courses. CUNY refers to these 
credits as “equated credits.” 

“Enrollment is based on courses that students are still enrolled in as of the end of the add/drop period. Full-time 
enrollment is defined as enrollment in 12 or more credits. Part-time enrollment is defined as enrollment in fewer 
than 12 credits. 


semesters, as well as on enrollment rates and credit attempts during the main session of the third 
semester of study. These findings suggest that the early positive effects presented in Tables 4 
through 6 for the full sample continue to grow throughout the second semester and on to the 
beginning of the third semester. 

Skipping to the second semester (including the main session and the intersession), 
ASAP’s estimated effect on full-time enrollment is 20.3 percentage points for the first cohort of 
students. 37 In terms of total credits attempted during the second semester, ASAP’s estimated 


37 This effect is very similar to the program’s estimated impact on full-time enrollment during the main 
session of the second semester for the full sample. 
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Evaluation of Accelerated Study in Associate Programs (ASAP) for 
Developmental Education Students 

Table 7 

Academic Outcomes: First, Second, and Third Semesters 
First Cohort 


Early Impacts Report 



Program 

Control 

Difference 

Standard 

Outcome 

Group 

Group 

(Impact) 

Error 


First semester 


Enrolled 3 (%) 


97.6 

97.5 

0.1 

1.7 

Full time 


97.6 

86.8 

10.8 *** 

2.9 

Part time 


0.0 

10.7 

-10 7 *** 

2.5 

Total credits attempted 


16.8 

14.4 

2 4 *** 

0.5 

College-level credits 


11.5 

10.7 

0.8 * 

0.5 

Developmental credits 


5.3 

3.7 

1 ^ *** 

0.3 

Total credits earned 


12.0 

9.5 

2 5 *** 

0.7 

College-level credits 


9.2 

7.7 

1 ^ *** 

0.6 

Developmental credits 


2.7 

1.8 

0 9 *** 

0.3 

Completed developmental requirements b (%) 

44.6 

30.8 

13.8 *** 

5.3 


Second semester 


Enrolled 3 (%) 

90.5 

78.6 

1 1 9 *** 

4.0 

Full time 

86.3 

66.0 

20 3 *** 

4.6 

Part time 

4.2 

12.6 

4 *** 

3.1 

Total credits attempted 

14.2 

10.8 

3 4 *** 

0.7 

College-level credits 

11.4 

9.1 

2 3 *** 

0.6 

Developmental credits 

2.8 

1.7 

1 2 *** 

0.3 

Total credits earned 

9.9 

7.9 

2 o *** 

0.7 

College-level credits 

oo 
o o 

7.3 

1.6 ** 

0.7 

Developmental credits 

i.i 

0.6 

0.5 ** 

0.2 

Cumulative first through second semester 

Total credits attempted 

31.0 

25.2 

5 8 *** 

1.0 

College-level credits 

22.9 

19.8 

3 i *** 

0.9 

Developmental credits 

8.1 

5.4 

2 7 *** 

0.5 

Total credits earned 

21.8 

17.3 

4 3 *** 

1.3 

College-level credits 

18.0 

14.9 

3 i *** 

1.1 

Developmental credits 

3.8 

2.4 

1 4 *** 

0.4 

Completed developmental requirements b (%) 

63.1 

38.4 

24 7 *** 

5.4 


(continued) 
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Table 7 (continued) 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

Standard 

Error 

Third semester 

Enrolled 3 (%) 

81.5 

70.4 

H I ** 

4.7 

Full time 

80.4 

57.9 

22.5 *** 

5.0 

Part time 

1.2 

12.6 

j 4 *** 

2.8 

Total credits attempted 

11.0 

8.9 

2 2 *** 

0.6 

College-level credits 

10.0 

8.0 

2 \ *** 

0.6 

Developmental credits 

1.0 

0.9 

0.1 

0.2 

Cumulative first through third semester 

Number of semesters enrolled 3 

2.7 

2.5 

Q 2 *** 

0.1 

Full time 

2.6 

2.1 

0 5 *** 

0.1 

Part time 

0.1 

0.4 

-0 3 *** 

0.1 

Number of sessions enrolled 3 

3.7 

3.0 

Q 2 *** 

0.1 

Sample size (total = 327) 

168 

159 




SOURCE: MDRC calculations from CUNY Institutional Research Database (IRDB). 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by site. 

First and second semester measures include main sessions and intersessions. Third semester measures only 
include the main session. 

“Developmental credits” are credits associated with developmental reading, writing, and math courses; English 
as a Second Language classes; and a small number of other non-college-level courses. CUNY refers to these 
credits as “equated credits.” 

Enrollment is based on courses that students are still enrolled in as of the end of the add/drop period. Full-time 
enrollment is defined as enrollment in 12 or more credits. Part-time enrollment is defined as enrollment in fewer 
than 12 credits. 

b Completion of developmental requirements is contingent upon passing CUNY Assessment Tests, passing the 
highest level of developmental education, and/or passing a college-level class in each subject. This measure 
includes students who passed CUNY Assessment Tests prior to the first semester. 

c During this period, students could enroll for a maximum of five sessions — three main sessions and two 
intersessions. 

impact is 3.4 credits for the first cohort of students. 38 In part as a result of students attempting 
more credits, ASAP had a positive effect on credits earned in the second semester. During 
this time period, the first cohort of ASAP students earned 9.9 credits on average. By compar- 
ison, their control group counterparts earned a total of 7.9 credits on average — for an 


38 For the first cohort, the estimated impact on total credits attempted during the main session of the second 
semester is 2.1 credits. The remaining impact of 1.3 credits occurred during the intersession of the second 
semester. (Not shown in tables.) 
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estimated impact of 2.0 credits. Summing the second semester impact with the first semester 
impact (2.5 credits), ASAP is estimated to have caused students to earn an additional 4.5 
total credits over two semesters. Unlike many single-semester programs whose positive 
effects are at best maintained over time and at worst fade, ASAP’s effects continued to grow 
during the second semester. 

Looking to the third semester, the program’s estimated impact on enrollment is again 
over 10 percentage points, and the estimated impact on full-time enrollment is again over 20 
percentage points. Given these enrollment numbers, it is unsurprising that program group 
students again attempted over two credits more than their control group counterparts during the 
third semester of study, and they are well-positioned to earn more credits. 

The findings presented in Table 7 should be considered with caution, since they repre- 
sent only one-third of the full ASAP sample. With that caveat in mind, they suggest that the 
early positive effects of ASAP continue to grow through at least three semesters of follow-up. 
Future MDRC reports will provide detailed findings for the full sample during additional 
semesters of follow-up, including information on graduation rates. 

Summary of Early Impacts 

During the first semester of study, ASAP helped students earn significantly more cred- 
its than they would have had they received their college’s usual services. During the second 
semester of study, the programs’ effects are on track to be even more dramatic than the effects 
observed in first semester of study. While MDRC’s evaluation of ASAP is still in an early stage, 
the evidence thus far indicates that ASAP has a very positive effect on students’ short-term 
academic progress. 


Conclusions and Next Steps 

The concluding section of this report puts the favorable short-term impacts for ASAP in the 
context of other higher education evaluations conducted by MDRC and looks ahead to next 
steps in this project. 

ASAP’s Impact Findings in the Context of MDRC’s Other Community 
College Research 

In order to provide perspective on ASAP’s early impact findings, it might be useful to 
situate them within the context of several other random assignment evaluations conducted at 
community colleges. Over the last decade, for example, MDRC has conducted 15 experimental 
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evaluations of programs intended to increase community college students’ academic success. 39 
These studies measured the impacts of an array of interventions, in a variety of settings, target- 
ing an assortment of different students, with a wide range of costs. Of the programs evaluated, 
half had positive statistically significant average impacts on total credits earned after the first 
semester of study, indicating that the programs were improving students’ progress towards a 
degree. Impact estimates (regardless of statistical significance) ranged from 0.0 to 1.4 total 
credits earned. During the first semester, ASAP’s impact on total credits earned was 2. 1 credits, 
150 percent as large as the next largest impact estimate MDRC has observed to date. Across 
those same studies, only two had statistically significant impacts on second semester enrollment 
rates. One had a 15 -percentage-point impact on second semester enrollment. Among the other 
studies, impacts ranged from approximately - 1 to 7 percentage points (regardless of statistical 
significance). The estimated impact of ASAP on second semester enrollment is 10 percentage 
points. While MDRC’s studies are not necessarily representative of all experimental research in 
community colleges, this comparison suggests that the early effects of ASAP are larger than 
what has been observed in many other rigorous evaluations of promising programs. 

It is also important to emphasize that most of ASAP’s supports and services last for 
three years. To the authors’ knowledge, this is longer than any community college program that 
has been evaluated using a random assignment research design. Will the early impacts be 
maintained? Might they grow larger? Will early impacts translate into impacts on degree 
completion, the main goal of ASAP? The answers to these questions are a critical part of the 
developing story about this program and will be a focus of future MDRC reports. 

Looking Ahead 

In the future, MDRC will share findings on longer-term academic outcomes, including 
two- and three-year graduation rates; a full implementation story; and infonnation on ASAP’s 
costs. If ASAP increases graduation rates substantially, MDRC may study the longer-term 
effects of ASAP on employment and earnings in a second phase of the study. 

The ASAP evaluation is not designed to disentangle, with great certainty, which com- 
ponents of the program contribute the most to the estimated impacts. 40 However, the implemen- 

39 These experiments include some studies of the same type of program. For example, MDRC has con- 
ducted experiments of seven variations of “learning communities” and four variations of performance-based 
scholarships. The results of these studies can be found in the following MDRC publications: Cha and Patel 
(2010); Miller, Binder, Harris, and Krause (2011); Richburg-Hayes et al. (2009); Richburg-Hayes, Sommo, 
and Welbeck (2011); Scrivener et al. (2008); Scrivener, Sommo, and Collado (2009); Scrivener and Weiss 
(2009); Visher, Butcher, and Cerna (2010); Visher and Teres (2011); Weiss, Visher, and Wathington (2010); 
Weissman et al. (2011); Weissman et al. (2012). 

40 This would have required a much larger sample size and the simultaneous implementation of different 
versions of the program. 
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tation research will shed light on that issue. Program group and control group students were 
surveyed about a year after they entered the study in order to gain a better understanding of the 
differences between the two groups’ experiences. It is these differences in experiences that are 
likely the cause of the observed positive effects of the program. The survey results will help 
answer questions like: How much more often, if at all, did program group students visit advisers 
compared with their control group counterparts? Did the content covered in advisement differ 
for the two groups of students? Similar questions regarding tutoring and career counseling will 
also be explored. Understanding how students’ experiences differed across the multiple dimen- 
sions of the program may begin to suggest which parts of the program made the biggest 
difference, and what other colleges would need to do if they want to try to replicate CUNY’s 
efforts. If certain elements of the program are essential and others are less important, a pared- 
down version of the program could be created that might yield similar effects. 

In addition to the student survey results, future reports will also present information 
from interviews with administrators and staff at the colleges — both those in ASAP and those 
who work with the general college population. Like the survey results, the interview data will 
help describe the experiences of program group and control group members. In addition, the 
interviews with the ASAP administrators and staff will also provide insight into how the 
program is executed by the participating colleges. 

Lastly, because ASAP includes multiple components and last for three years, the addi- 
tional cost of ASAP compared with the colleges’ regular services is almost certainly higher than 
that of other less comprehensive programs. But how much more expensive is ASAP than the 
colleges’ usual services? This infomiation will be important for interpreting the program’s 
positive effects and for considering its replication at other institutions. 

It is clear that ASAP’s package of requirements, messages, services, and financial bene- 
fits can improve short-term academic outcomes for students. If ASAP generates positive 
changes for students over a longer timeframe, it will provide important lessons for college 
administrators and policymakers. ASAP’s lessons should be especially relevant for other bold, 
comprehensive refonns, such as the Gates Foundation’s Completion by Design initiative, which 
aims to help groups of community colleges refonn the educational experience from entry to 
graduation. The ASAP evaluation will provide critical information on how much difference a 
long-lasting, multifaceted intervention can make for community college students. In the 
meantime, based on ASAP’s effects for students to date — those in the MDRC random assign- 
ment study as well as students in prior entering cohorts — CUNY is expanding the program to 
serve over 4,000 students by 2014, three times its current size. 
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by Research Group 




Evaluation of Accelerated Study in Associate Programs (ASAP) for 
Developmental Education Students 

Appendix Table 1 

Selected Characteristics of Sample Members at Baseline, by Research Group 

Early Impacts Report 



Full 

Sample 

Program 

Group 

Control 

Group 

Gender (%) 

Male 

37.9 

36.1 

39.8 

Female 

62.1 

63.9 

60.2 

Age (%) 

Less than 20 years 

57.1 

56.8 

57.5 

20-24 years 

26.0 

25.3 

26.7 

25 years and over 

16.9 

18.0 

15.7 

Average age (years) 

21.5 

21.6 

21.3 

Marital status (%) 

Married 

6.1 

6.0 

6.3 

Unmarried 

78.6 

78.0 

79.1 

Missing 

15.3 

16.0 

14.6 

Lives with parents (%) 

73.7 

73.9 

73.6 

Parents pay more than half of expenses (%) 

41.0 

41.7 

40.2 

Missing 

18.0 

17.1 

18.9 

Race/Ethnicity a (%) 

Hispanic 

43.6 

44.5 

42.7 

White 

10.0 

10.4 

9.6 

Black 

34.3 

32.7 

35.9 

Asian or Pacific Islander 

7.5 

7.1 

7.9 

Other b 

4.6 

5.3 

4.0 

Has one or more children (%) 

15.3 

17.4 

13.1 * 

Currently employed (%) 

31.3 

30.1 

32.6 

Among those currently employed: 

Number of hours worked per week in current job (%) 

1-10 hours 

8.1 

9.7 

6.7 

11-20 hours 

34.0 

37.1 

31.1 

21-30 hours 

31.7 

23.4 

39.3 

31-40 hours 

24.7 

29.0 

20.7 

More than 40 hours 

1.5 

0.8 

2.2 

Highest grade completed (%) 

10th grade or lower 

7.3 

6.7 

7.9 

1 1th grade 

7.8 

8.4 

7.2 

12 th grade c 

75.9 

76.1 

75.7 

Missing 

9.0 

8.9 

9.2 


(continued) 
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Appendix Table 1 (continued) 



Full 

Program 

Control 


Sample 

Group 

Group 

Diplomas/degrees eamed d (%) 




High school diploma 

73.8 

74.1 

73.5 

General Educational Development (GED) certificate 

20.8 

20.6 

21.1 

Occupational/technical certificate 

5.6 

4.5 

6.6 

Other 

1.7 

1.6 

1.8 

None 

6.0 

5.9 

6.2 

Date of high school graduation/GED receipt (%) 




During the past year 

49.4 

51.4 

47.4 

Between one and two years ago 

13.3 

12.6 

13.9 

Between two and five years ago 

13.1 

12.6 

13.5 

More than five years ago 

13.2 

13.1 

13.3 

Has not earned a diploma/GED c 

6.0 

5.8 

6.3 

Missing 

5.0 

4.4 

5.6 

Student's status 6 (%) 




Incoming freshman 

60.0 

62.7 

57.3 

Returning student 

33.5 

31.9 

35.1 

Transfer student 

6.5 

5.3 

7.6 

Highest degree student plans to attain (%) 




Associate's 

2.8 

3.0 

2.6 

Bachelor's 

31.4 

32.6 

30.2 

Master's 

41.6 

41.3 

41.9 

Professional or doctorate 

17.8 

17.4 

18.3 

Beyond an associate's, unspecified 

6.4 

5.7 

7.0 

First person in family to attend college (%) 

30.3 

28.1 

32.5 

Highest degree/diploma earned by mother (%) 




Not a high school graduate 

19.5 

18.6 

20.4 

High school diploma or GED 

21.8 

21.5 

22.0 

Some college, did not complete a degree 

16.0 

16.0 

16.0 

College degree (AA, BA, MA, PhD) 

18.6 

18.4 

18.9 

Missing 

24.1 

25.5 

22.7 

Language other than English spoken regularly in home (%) 

44.7 

45.3 

44.2 

Sample size 

896 

451 

445 

(continued) 
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Appendix Table 1 (continued) 

SOURCE: MDRC calculations using Baseline Information Form (BIF) data. 

NOTES: To analyze whether baseline characteristics jointly predicted research group status, a likelihood ratio 
test was performed. This yielded a p-value of 0.91 . Convention suggests that these probabilities are large 
enough that these potential differences can be ignored in the analyses. 

A two-tailed t-test was applied to differences between the program group and control group for variables 
that are not mutually exclusive and mutually exhaustive (e.g., diplomas/degrees earned). A chi-squared test was 
applied to differences between the groups of categorical variables that are mutually exclusive and mutually 
exhaustive (e.g., race/ethnicity). Levels for statistically significant differences between program and control 
groups are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. 

Italic type indicates nonexperimental data. Significance tests are not calculated for nonexperimental data. 
Missing values are only included in variable distributions for characteristics with more than 5 percent of the 
sample missing. 

Distributions may not add to 100 percent because of rounding. 

a Respondents who said they are Hispanic and chose a race are included only in the Hispanic category. 
Respondents who said they are not Hispanic and chose more than one race are included in the Other category. 
b Other includes multi-racial, Native American/Alaska Native, and other race/ethnicities. 
c This includes students who were currently enrolled in high school at study intake. 
d Distributions may not add to 100 percent because categories are not mutually exclusive. 
e Studenf s status at the start of first semester in the study. 
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About MDRC 


MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 
to learning what works to improve the well-being of low-income people. Through its research 
and the active communication of its findings, MDRC seeks to enhance the effectiveness of so- 
cial and education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to leam not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy are- 
as and target populations. Once known primarily for evaluations of state welfare-to-work pro- 
grams, today MDRC is also studying public school refonns, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Children’s Development 

• Improving Public Education 

• Raising Academic Achievement and Persistence in College 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 



